Hadoop NameNode 启动不了

Posted by Jackson on 2017-08-15

报错信息为:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/home/hadoop/tmp/dfs/name is in an inconsistent state: storage directory does not exist
or is not accessible.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:377)
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:228)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:799)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:614)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:676)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:844)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:823)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1547)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)

报错信息为:InconsistentFSStateException: Directory /home/hadoop/tmp/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.

所做的操作是强行使用 kill -9 杀掉了NameNode 的进程

解决的方案是:

1.stop hadoop的所有服务

2.在hadoop的根目录执行./bin/hadoop namenode -format 格式化NameNode

3.重启HDFS和Yarn

此时又发现DataNode启动不了,去日志中查看DataNode的日志,发现报错信息如下

1
2
3
ARN org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: 
Unexpected error trying to delete block BP-196862025-192.168.52.50-1575212732331
blk_1073741921_1097 at file /home/hadoop/tmp/dfs/data/current/BP-196862025-192.***.***.***-1575212732331/current/finalized/subdir0/subdir0/blk_1073741921. Ignored.

上述日志的意思是删除文件时候未删除成功,此时我们手动进入该文件进行删除文件内容。

然后重启hadoop,成功解决。

总结:当Hadoop集群出现问题时候,先去日志里面找对应的Error,分析Error出现的原因从而找到对应的解决方法。